A number of different video encoding standards have been established for encoding digital video sequences. The Moving Picture Experts Group (MPEG), for example, has developed a number of standards including MPEG-1, MPEG-2 and MPEG-4. Other examples include the International Telecommunication Union (ITU) H.263 standard, and the emerging ITU H.264 standard. These video encoding standards generally support improved transmission efficiency of video sequences by encoding data in a compressed manner. Compression reduces the overall amount of data that needs to be transmitted for effective transmission of video frames.
The MPEG-4, ITU H.263 and ITU H.264 standards, for example, support video encoding techniques that utilize similarities between successive video frames, referred to as temporal or Inter-frame correlation, to provide Inter-frame compression. The Inter-frame compression techniques exploit data redundancy across frames by converting pixel-based representations of video frames to motion representations. Frames encoded using Inter-frame techniques are referred to as P (“predictive”) frames or B (“bi-directional”) frames. Some frames, referred to as I (“intra”) frames, are encoded using spatial compression, which is non-predictive.
In order to meet low bandwidth requirements, some video applications, such as video telephony or video streaming, reduce the bit rate by encoding video at a lower frame rate using frame skipping. An intentionally skipped frame may be referred to as an “S” (“skipped”) frame. Unfortunately, low frame rate video can produce artifacts in the form of motion jerkiness. Therefore, frame interpolation, also known as frame rate up conversion (FRUC), is typically used at the decoder to interpolate the content of intentionally skipped frames.
A variety of FRUC techniques have been developed, and can be divided into two categories. A first FRUC category includes frame repetition (FR) and frame averaging (FA), which both use a combination of video frames without consideration of motion. These algorithms provide acceptable results in the absence of motion. When there is significant frame-to-frame motion, however, FR tends to produce motion jerkiness, while FA produces blurring of objects.
A second FRUC category relies on advanced conversion techniques that employ motion. In this category, the quality of an interpolated frame depends on the difference between estimated motion and true object motion. In typical FRUC applications, the decoder obtains motion information for an interpolated frame from motion information for adjacent frames. However, the motion vectors of an S frame that are obtained directly from the motion vectors of adjacent frames are not sufficiently accurate and can result in various artifacts in interpolated frames. Also, for Intra-coded blocks, there is no motion information available to use for interpolation of S frames, generally requiring additional estimation or processing overhead at the decoder for reliable interpolation.